Detecting Malicious Software by Monitoring Anomalous Windows Registry Accesses

We present a host-based intrusion detection system for Microsoft Windows. The core of the system is an algorithm that detects attacks on a host machine by looking for anomalous accesses to the Windows Registry. The key idea is to first train a model of normal registry behavior for a host and to use this model to detect abnormal registry accesses at run-time. The system trains a normal model using data that contains no attacks and then at run-time checks each access to the registry in real time to determine whether or not the behavior is abnormal and corresponds to an attack. We evaluate the system by training the system on a set of normal registry accesses and then use the system to detect the actions of malicious software. The system is effective in detecting the actions of malicious software while maintaining a low rate of false alarms.